Determining Subgroup Difference Importance with Complex Survey Designs An Application of Weighted Dominance Analysis

Objective: Determining which subgroups show the most substantial differences on a measure is a common use of surveys. How to accurately and fairly determine which subgrouping is most important has not been addressed adequately in the literature. I show how dominance analysis is a useful way to identify the most important subgroup differences. Because surveys commonly employ complex sampling designs, I also provide practical guidelines for determining subgroup relative importance from complex survey data. Methods: The advantages of dominance analysis over alternative analysis procedures for determining importance are discussed using an empirical example from the political party affiliation question in the General Social Survey. Additionally, a Monte Carlo simulation was conducted to examine the accuracy of dominance analysis with complex sampling accounting for sample weights, strata, both, and neither compared to known population values. Results: Dominance analysis clearly identifies the urbanicity subgrouping as having the most important differences on political party affiliation. Results also show the use of survey weights can have non-trivial effects on subgroup rank ordering. The simulation shows that weighed dominance statistics were more accurate than unweighted statistics. Stratified analyses had no effect on relative importance statistics. Conclusions: Dominance analysis is a useful way to identify key subgroup differences on survey measures. Survey weights are necessary to use for dominance analysis, when available, in order to obtain an accurate representation of the rank order and magnitude of differences between subgroup indicators on a survey measure. The article concludes by outlining situations where dominance analysis is recommended. 2 Joseph Nicholas Luchman